METHOD FOR QUANTITATIVE VIDEO-MICROSCOPY AND ASSOCIATED 
SYSTEM AND COMPUTER SOFTWARE PROGRAM PRODUCT 



FIELD OF THE INVENTION 
The present invention relates to image analysis and, more particularly, to a 
method for quantitative video-microscopy in cellular biology and pathology applications 
and an associated system and computer software program product therefor. 

BACKGROUND OF THE INVENTION 
Effective analysis of microscopic images is essential in cellular biology and 
pathology, particularly for detection and quantification of genetic materials such as, for 
example, genes or messenger RNA, or the expression of this genetic information in the 
form of proteins such as through, for example, gene amplification, gene deletion, gene 
mutation, messenger RNA molecule quantification, or protein expression analyses. Gene 
amplification is the presence of too many copies of the same gene in one cell, wherein a 
cell usually contains two copies, otherwise known as alleles, of the same gene. Gene 
deletion indicates that less than two copies of a gene can be found in a cell. Gene 
mutation indicates the presence of incomplete or non-functional genes. Messenger RNAs 
(mRNA) are molecules of genetic information, synthesized from a gene reading process, 
that serve as templates for protein synthesis. Protein expression is the production of a 
given protein by a cell. If the gene coding for the given protein, determined from a 
protein expression process, is enhanced or excess copies of the gene or mRNA are 
present, the protein may be over-expressed. Conversely, if the gene coding is suppressed 
or absent, the protein may be under-expressed or absent. 

Normal cellular behaviors are precisely controlled by molecular mechanisms 
involving a large number of proteins, mRNAs, and genes. Gene amplification, gene 
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deletion, and gene mutation are known to have a prominent role in abnormal cellular 
behaviors through abnormal protein expression. The range of cellular behaviors of 
concern includes behaviors as diverse as, for example, proliferation or differentiation 
regulation. Therefore, effective detection and quantification in gene amplification, 
5 deletion and mutation, mRNA quantification, or protein expression analyses is necessary 
in order to facilitate useful research, diagnostic and prognostic tools. 

There are numerous laboratory techniques directed to detection and quantification 
in gene amplification, deletion and mutation, mRNA quantification, or protein expression 
analyses. For example, such techniques include Western, Northern and Southern blots, 
10 polymerase chain reaction ("PCR"), enzyme-linked immunoseparation assay ("ELISA"), 
and comparative genomic hybridization ("CGH") techniques. However, microscopy is 
routinely utilized because it is an informative technique, allowing rapid investigations at 
the cellular and sub-cellular levels while capable of being expeditiously implemented at a 
relatively low cost. 

15 When microscopy is the chosen laboratory technique, the biological samples must 

first undergo specific detection and revelation preparations. Once the samples are 
prepared, a human expert typically analyzes the samples with a microscope alone in a 
qualitative study, or with a microscope coupled to a camera and a computer in a 
quantitative and generally standardized study. In some instances, the microscope may be 

20 configured for fully automatic analysis, wherein the microscope is automated with a 
motorized stage and focus, motorized objective changers, automatic light intensity 
controls and the like. 

The preparation of the samples for detection may involve different types of 
preparation techniques that are suited to microscopic imaging analysis, such as, for 

25 example, hybridization-based and immunolabeling-based preparation techniques. Such 
detection techniques may be coupled with appropriate revelation techniques, such as, for 
example, fluorescence-based and visible color reaction-based techniques. 

In Situ Hybridization ("ISH") and Fluorescent In Situ Hybridization ("FISH") are 
detection and revelation techniques used, for example, for detection and quantification in 

30 genetic information amplification and mutation analyses. Both ISH and FISH can be 
applied to histological or cytological samples. These techniques use specific 
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complementary probes for recognizing corresponding precise sequences. Depending on 
the technique used, the specific probe may include a chemical (ISH) marker or a 
fluorescent (FISH) marker, wherein the samples are then analyzed using a transmission 
microscope or a fluorescence microscope, respectively. The use of a chemical marker or 
5 a fluorescent marker depends on the goal of the user, each type of marker having 
corresponding advantages over the other in particular instances. 

In protein expression analyses, immunohistochemistry ("IHC") and 
immunocytochemistry ("ICC") techniques, for example, may be used. EHC is the 
application of immunochemistry to tissue sections, whereas ICC is the application of 

10 immunochemistry to cultured cells or tissue imprints after they have undergone specific 
cytological preparations such as, for example, liquid-based preparations. 
Immunochemistry is a family of techniques based on the use of a specific antibody, 
wherein antibodies are used to specifically target molecules inside or on the surface of 
cells. The antibody typically contains a marker that will undergo a biochemical reaction, 

15 and thereby experience a change color, upon encountering the targeted molecules. In 
some instances, signal amplification may be integrated into the particular protocol, 
wherein a secondary antibody, that includes the marker stain, follows the application of a 
primary specific antibody. 

In both hybridization and immunolabeling studies, chromogens of different colors 

20 are used to distinguish among the different markers. However, the maximum number of 
markers that may be used in a study is restricted by several factors. For example, the 
spectral overlapping of the colors used to reveal the respective markers maybe a limiting 
factor because dyes may absorb throughout a large portion of the visible spectrum. 
Accordingly, the higher the number of dyes involved in a study, the higher the risk of 

25 spectral overlapping. Further, the spectral resolution of the acquisition device may be a 
limiting factor and the minimal color shift that the device is able to detect must be 
considered. 

In addition, immunochemistry, as well as chemistry in ISH, are generally 
considered to exhibit poor sensitivity when quantification of a marker must be achieved. 
30 However, the quantification accuracy of these techniques may be dependent upon several 
factors. For instance, the type of reaction used may play a role in the accuracy of the 
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technique since the linearity of the relationship between ligand concentration and the 
degree of the immunochemical staining reaction may strongly depend on the reaction 
type. More particularly, for example, a peroxidase / anti-peroxidase method may be more 
linear than a biotin-avidin method. The cellular localization of the markers may also 

5 affect accuracy where, for example, if membrane and nuclear markers spatially overlap, 
the resulting color is a mixture of the respective colors. Accordingly, since the 
corresponding quantification is subjective, the accuracy of the determination may be 
affected. In addition, a calibration standard such as, for example, cells with known 
features, gels with given concentrations of the marker, or the like, may be required where 

10 a developed analysis model is applied to a new and different case. Staining kits are 
generally available which incorporate calibration standards. However, the calibration 
standard is usually only applicable to a particular specimen, such as a specific cell or a 
structure of a specific type which is known to exhibit constant features with respect to the 
standard, and may be of limited utility when applied to a sample of a different nature. 

1 5 Overall, the described "colorimetric" studies present sample analysis information 

in color and facilitate processing and quantification of the information to thereby help to 
provide a diagnosis or to form a prognosis of the particular case. For illustration, the 
detection and quantification of the HER2 protein expression and/or gene amplification 
may be assessed by different approaches used in quantitative microscopy. HER2 is a 

20 membrane protein that has been shown to have a diagnostic and prognostic significance 
in metastatic breast cancer. Because HER2 positive patients were shown to be more 
sensitive to treatments including Herceptin® (a target treatment developed by 
Genentech), the definition of the HER2 status of metastatic breast cancers has been 
proven to be of first importance in the choice of the appropriate treatment protocol. This 

25 definition of the HER2 status was based on a study of samples treated with either 
hybridization (FISH, ISH) or immunolabeling (IHC) techniques. 

In such studies, using FISH with, for example, an FDA approved kit such as 
PathVysion® produced by Vysis, requires an image analysis protocol for counting the 
number of copies of the HER2 gene present in every cell. In a normal case, two copies of 

30 the gene are found in each cell, whereas more than three copies of the gene in a cell 

indicate that the gene is amplified. Alternatively, using IHC with, for example, an FDA 
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approved kit such as Herceptest® produced by Dako, requires an image analysis protocol 
that classified the cases into four categories depending on the intensity and localization of 
the HER2 specific membrane staining. Current studies tend to show that these two 
investigation techniques (hybridization and immunolabeling) maybe complementary and 

5 may help pathologists in tumor sub-type diagnosis when combined. 

However, such colorimetry studies require extensive sample preparation and 
procedure control. Thus, when disposing of adapted staining protocols, it is critical to be 
able to verify that the staining for each sample matches the particular model used in the 
image acquisition and processing device such that useful and accurate results are obtained 

1 0 from the gathered information. Otherwise, the analysis may have to be repeated, starting 
again from the sample preparation stage, thereby possibly resulting in a costly and time- 
consuming process. 

In a typical microscopy device based on image acquisition and processing, the 
magnified image of the sample must first be captured and digitized with a camera. 

15 Generally, charge coupled device (CCD) digital cameras are used in either light or 
fluorescence quantitative microscopy. Excluding spectrophotometers, two different 
techniques are generally used to perform such colorimetric microscopy studies. In one 
technique, a black and white (BW) CCD camera may be used. In such an instance, a gray 
level image of the sample is obtained, corresponding to a monochromatic light having a 

20 wavelength specific to the staining of the sample to be analyzed. The specific 

wavelength of light is obtained either by filtering a white source light via a specific 
narrow bandwidth filter, or by directly controlling the wavelength of the light source, 
using either manual or electronic controls. Accordingly, using this technique, the 
analysis time increases as the number of colors increases because a light source or a filter 

25 must be selected for every different sample staining or every different wavelength. 
Therefore, many different images of the sample, showing the spectral response of the 
sample at different wavelengths, must be individually captured in a sequential order to 
facilitate the analysis. When multiple scenes or fields of view must be analyzed, the 
typical protocol is to automate the sequence in a batch mode to conserve processing time. 

30 According to a second technique, a color CCD digital camera is used, wherein 

three gray level images of the sample are simultaneously captured and obtained. Each 
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gray level image corresponds to the respective Red, Green and Blue channel (RGB) of 
the color CCD camera. The images are then analyzed directly in the RGB color space by 
restricting the analysis to pixels located in a specific region of the RGB cube, the specific 
region also including pixels from a corresponding training database. Alternatively, the 
5 images are analyzed, after mathematical transform of the RGB color space, in one of the 
many color spaces defined by the CIE (International Commission on Illumination) such 
as, for example, an HLS (Hue, Luminance or Saturation) space. Alternatively, some 
camera manufacturers produce specific CCD cameras, wherein narrow bandwidth filters 
for targeting specific wavelengths may replace the usual Red, Green and Blue filters. In 

10 such an instance, the camera allows a fast image capture of the three spectral components 
of a scene in a parallel manner. However, cameras modified in this manner may be 
restricted to specific spectral analysis parameters because the filters cannot be changed 
and therefore cannot be adapted to address a unique dye combination used for the sample. 
Thus, the second technique generally relies upon either the detection of contrast between 

15 the specie/species of interest and the remainder of the sample or the analysis of the 
sample over a narrow bandwidth. 

Accordingly, techniques used in colorimetric analyses of prepared samples are of 
limited use in the detection and quantification of species of interest due to several factors 
such as, for example, spectral overlapping, mixing of colors due to spatially overlap of 

20 membrane, cytoplasmic, and nuclear markers, chromatic aberrations in the optical path, 
limited spectral resolution of the acquisition device, calibration particularities, 
subjectivity of the detection and quantification process, and inconsistencies between 
human operators. The image processing portion of colorimetric analysis techniques has 
historically been directed to the subjective detection of contrast within the prepared 

25 sample or to a complex and voluminous analysis of the sample at various specific 

wavelengths of light using a combination of light sources and filters. Therefore, there 
exists a need for a simpler and more effective colorimetric analysis technique that 
overcomes detection and quantification limitations found in prior art analysis techniques. 
Such a technique should also be capable of providing high quality data, comprising the 

30 necessary analysis information about the sample, while reducing subjectivity and 
inconsistency in the sample analysis. 
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SUMMARY OF THE INVENTION 
The above and other needs are met by the present invention which, in one 
embodiment, provides a method of determining an amount of at least one molecular 
specie comprising a sample, each molecular specie being indicated by a dye. The amount 
5 of the molecular specie is determined from an image of the sample captured as image 
data by a color image acquisition device in a video-microscopy system. First, an optical 
density of the sample is determined in each of a red, green, and blue channel at a 
particular pixel in the image. A corresponding optical density matrix is thereafter formed 
for that pixel. The optical density matrix is then multiplied by the inverse of a relative 
10 absorption coefficient matrix so as to form a resultant matrix for the pixel. The relative 
absorption coefficient matrix comprises a relative absorption coefficient for each dye, 
independently of the sample, in each of the red, green, and blue channels. The resultant 
matrix thus comprises the amount of each molecular specie, as indicated by the respective 
dye, for that pixel. 

15 The amount of each molecular specie is determined from a color image of the 

sample. According to one embodiment of the present invention, a color image 
acquisition device, such as an RGB camera and associated frame grabber or a color 
scanner, is used to acquire the image of the sample. The image may then be balanced and 
normalized according to an empty field (white) reference and a black field image and, in 

20 some instances, corrected for shading. The image also corrected for chromatic 

aberrations on a channel by channel basis. Subsequently, an optical density of the sample 
is determined in each of the red, green, and blue channels at a particular pixel in the 
image from the measured transmitted light. A corresponding optical density matrix is 
thereafter formed for that pixel and then multiplied by the inverse of a relative absorption 

25 coefficient matrix of the dyes present in the sample so as to form a resultant matrix for 
the pixel representing the optical density contributions from each dye. Since the relative 
absorption coefficient matrix comprises a relative absorption coefficient for each of the 
dyes used in the sample preparation protocol in each of the red, green, and blue channels, 
the resultant matrix thus comprises the amount, as expressed in proportion to. 

30 concentration, of each molecular species as indicated by the respective dyes for that 
pixel. 
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Another advantageous aspect of the present invention comprises a video- 
microscopy system for determining an amount of a molecular specie comprising a sample 
from an image of the sample, wherein each molecular specie is indicated by a dye. The 
system comprises a color image acquisition device capable of capturing a magnified 
5 digital image of the sample as image data, and a computer device operably engaged with 
the image acquisition device. The computer device comprises a processing portion 
configured to determine an optical density of the sample from the image data in each of a 
red, green, and blue channel and at a pixel in the image to thereby form a corresponding 
optical density matrix for the pixel. Another processing portion of the computer device is 

1 0 further configured to multiply the optical density matrix by the inverse of a relative 

absorption coefficient matrix so as to form a resultant matrix for the pixel of the image. 
The relative absorption coefficient matrix comprises a relative absorption coefficient for 
each dye, independently of the sample, in each of the red, green, and blue channels. The 
resultant matrix thus comprises the amount of each molecular specie, as indicated by the 

1 5 respective dye, for that pixel. 

Still another advantageous aspect of the present invention comprises a computer 
software program product configured to be executable on a computer device and capable 
of determining an amount of a molecular specie comprising a sample from a digital 
image of the sample captured as image data by a color image acquisition device in a 

20 video-microscopy system, wherein each molecular specie is indicated by a dye. One 
executable portion of the computer software program product is capable of determining 
an optical density of the sample in each of a red, green, and blue channel and at a pixel in 
the image to thereby form a corresponding optical density matrix for the pixel. Another 
executable portion of the computer software program product is further capable of 

25 multiplying the optical density matrix by the inverse of a relative absorption coefficient 
matrix so as to form a resultant matrix for the pixel of the image. The relative absorption 
coefficient matrix comprises a relative absorption coefficient for each dye, independently 
of the sample, in each of the red, green, and blue channels. The resultant matrix thus 
comprises the amount of each molecular specie, as indicated by the respective dye, for 

30 that pixel. 
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Such imaging techniques as described herein, otherwise known as multi-spectral 
imaging techniques, when particularly adapted to color imaging, allow a substantially 
real time, or video rate, processing and viewing of the sample. The use of, for example, a 
RGB color CCD camera allows acquisition and processing time for sample images to be 

5 performed at a video rate, typically 40 millisecond per frame, which provides a 

considerable advantage as compared to prior art imaging techniques which generally 
exhibit field of view acquisition and processing times of over 1 second. Where an RGB 
camera is used by the system, image acquisition through the different channels is 
performed in parallel and look-up tables (LUT) can be generated so as to map the 

1 0 possible RGB color input values to predetermined concentrations and/or transmittance of 
each of various dyes. Thus, such capabilities may, for example, enhance processing 
speed and facilitate real time processing for display purposes. 

Accordingly, an a posteriori evaluation of the image can be performed to evaluate 
the efficiency of an a priori known dye combination used to solve the linear equations for 

1 5 each pixel That is, an evaluation as detailed herein also provides a confidence evaluation 
for each pixel in that the color and intensity measured at the given pixel may be justified 
by a combination of the a priori known dyes. Such a confidence evaluation can be 
expressed as the inverse to the closest match in the theoretical model. In a situation in 
which there are- fewer dyes to evaluate than input channels (less unknown parameters 

20 than equations, e.g. one counter stain + one marker (= 2 dyes) when 3 input channels 

(RGB) are available), the redundant information may be used to minimize potential errors 
and therefore to maximize the accuracy of the detection and quantification system. 

Thus, embodiments of the present invention comprise a colorimetric analysis 
technique for prepared samples that provides effective detection and quantification of 

25 species of interest that overcomes limiting factors of prior art techniques such as, for 

example, spectral overlapping, mixing of colors due to spatially overlap of membrane and 
nuclear markers, limited spectral resolution of the acquisition device, calibration 
particularities, the subjectivity of the detection and quantification process, and 
inconsistencies between human operators of the analysis equipment. Embodiments of the 

30 present invention further provide an image processing technique which does not rely 
upon the subjective detection of contrast within the prepared sample or a complex and 
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voluminous analysis of the sample at specific wavelengths of light using a combination 
of light sources and filters. Therefore, embodiments of the present invention provide a 
simpler and more effective colorimetric analysis technique that overcomes detection and 
quantification limitations in prior art analysis techniques, reduces subjectivity and 
5 inconsistency in the sample analysis, and is capable of providing the necessary analysis 
information about the sample, once an image of the sample is captured, without relying 
upon further examination of the sample to complete the analysis. These and other 
advantages are realized over prior art colorimetric analysis techniques as described 
herein. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 
Having thus described the invention in general terms, reference will now be made 
to the accompanying drawings, which are not necessarily drawn to scale, and wherein: 
FIG, 1 is a general schematic representation of a quantitative video-microscopy 
1 5 system according to one embodiment of the present invention. 

FIG. 2 is a schematic representation of the practical realization in an extended 
configuration of a quantitative video-microscopy system according to one embodiment of 
the present invention. 

20 DETAILED DESCRIPTION OF THE INVENTION 

The present invention now will be described more fully hereinafter with reference 

to the accompanying drawings, in which preferred embodiments of the invention are 

shown. This invention may, however, be embodied in many different forms and should 

not be construed as limited to the embodiments set forth herein; rather, these 
25 embodiments are provided so that this disclosure will be thorough and complete, and will 

fully convey the scope of the invention to those skilled in the art. Like numbers refer to 

like elements throughout. 

The platform for the evaluation of biological samples via image analysis is 

increasingly shifting from a general-purpose image analyzer to a more, and often highly, 
30 specialized dedicated "pathology workstation." Such workstations are typically designed 

to facilitate routine work, often combining many of the tools needed to provide a 
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pathologist with the necessary information to determine the best possible results. One 
example of such a workstation is illustrated in FIG. 1 as a quantitative video-microscopy 
system, indicated by the numeral 100, according to one embodiment of the present 
invention. The system 100 generally comprises a microscope 150 having a light source 

5 200 and a magnifying objective 250, a camera 300, a computer device 350, and a data 
transmission link 400 between the camera 300 and the computer device 350. The 
microscope 150 may comprise, for example, an Axioplan (or Axiovert) microscope 
produced by ZEISS of Germany or a similar microscope having a bright field light 
source. The camera 300 operably engages the microscope 150 and, in one embodiment, 

10 comprises a 3 CCD RGB camera such as, for instance, a Model No. DC-330E Dage-MTI 
RGB 3 CCD camera produced by Dage-MTI, Inc. of Michigan City, IN or a similar RGB 
camera. Typically, such a camera 300 also includes an associated frame grabber (not 
shown) to facilitate image capture, both the camera 300 and associated frame grabber 
being referred to herein as the "camera 300" for convenience. In some instances, both 

1 5 camera 300 and microscope 150 may be replaced by, for example, a linear flat scanner 
having a 3 CCD chip or equivalent and a controlled illumination source. For instance, a 
Model No. Super CoolScan 4000 ED scanner produced by Nikon Corporation may be 
used for low-resolution imaging. Note that, though different configurations of the 
necessary system 100 are contemplated by the present invention, the present invention 

20 will be described herein in terms of a camera 300 and associated microscope 150. 

Accordingly, one skilled in the art will understand and appreciate the capabilities and 
methodologies associated with these different configurations for accomplishing the 
present invention as detailed herein. 

The camera 300 is generally configured to capture an image 450 of a sample 500 

25 through the magnifying objective 250 (where a flat scanner is used, the image 450 is 
captured through internal lenses), wherein the image 450 may further comprise a digital 
image having corresponding image data (collectively referred to herein as "the image 
450"). The image 450 is generally captured as a whole, wherein the corresponding image 
data comprises a red channel 550, a green channel 600, and a blue channel 650 image of 

30 the field of view. The data transmission link 400 is configured so as to be capable of 

transmitting the image 450 to the computer device 350, wherein the computer device 350 
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is ftirther configured to be capable of analyzing the image 450 with respect to each of the 
red 550, green 600, and blue 650 channels. 

According to a particularly advantageous aspect of the present invention, the 
system 100 is configured to analyze the sample in accordance with the Lambert-Beer law. 
5 The Lambert-Beer law generally describes a proportionality that can be observed 

between the concentration of molecules in a solution (the concentration of the "molecular 
specie" or the "sample") and the light intensity measured through the solution. The 
Lambert-Beer law is typically expressed as: 

OD = elC (1) 

where OD is the optical density of the solution, € is a proportionality constant called the 

10 molar extinction or absorption coefficient, 1 is the thickness of the sample, and C is the 
concentration of the molecular specie. The absorption coefficient € is specific to the 
molecular specie and is typically expressed in units of L-mol'^cm' 1 . 

This proportionality relationship defined by the Lambert-Beer law has been 
verified under the several conditions including, for example, monochromatic light 

1 5 illuminating the sample, low molecular concentration within the sample, generally no 

fluorescence or light response heterogeneity (negligible fluorescence and diffusion) of the 
sample, and lack of chemical photosensitivity of the sample. Further, another 
requirement for an analysis according to the Lambert-Beer law includes, for instance, 
correct Koehler illumination of the sample under the microscope. Koehler illumination is 

20 available with many modern microscopes, providing an even illumination of the sample 
in the image plane and allowing for effective contrast control. Koehler illumination is 
critical for certain processes such as, for example, densitometry analysis. Correct 
Koehler illumination is typically provided by, for example, a two-stage illuminating 
system for the microscope in which the source is imaged in the aperture of the sub-stage 

25 condenser by an auxiliary condenser. The sub-stage condenser, in turn, forms an image 
of the auxiliary condenser on the object. An iris diaphragm may also be placed at each 
condenser, wherein the first iris controls the area of the object to be illuminated, and the 
second iris varies the numerical aperture of the illuminating beam. 
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The Lambert-Beer law has an additive property such that, if the sample comprises 
several light-absorbing molecular species, for example, si and s 2 , having respective 
concentrations Ci and C 2? the OD of a sample of thickness 1 (where U = 1 2 = 1 for the 
sample, as indicated in the solution hereinafter) can be expressed as: 

OD = 61 • h • Ci + 62 ' h • C 2 (2) 

5 This situation may occur, for example, in a biological analysis where a "scene," a field of 
view, or a portion of the sample has been stained with two dyes consisting of a marker 
dye for targeting the molecular specie of interest and a counterstain for staining the 
remainder of the sample. 

In order to accurately measure the concentration of given species imaged under a 

10 microscope, the measurements of the optical densities performed at different wavelengths 
must specifically correspond to the observed portion of the sample. That is, the 
microscopy system must be corrected for chromatic aberration, wherein such a correction 
or compensation may be accomplished by hardware, software, or a combination of 
software and hardware. Generally, glass tends to disperse light, which typically causes a 

15 simple glass lens to, for example, focus blue light at a shorter distance than red light. 
That is, a simple glass lens will exhibit different focal lengths for light comprising 
different wavelengths. This dispersion characteristic of glass gives rise to two observed 
effects. First, longitudinal chromatic aberration, or the positional difference of the focal 
points for different wavelengths of light along the vertical axis, is observed where, upon 

20 focusing the image for selected wavelengths of light corresponding to a particular color, 
the image will tend to be slightly out of focus when viewed under wavelengths of light 
corresponding to other colors. For example, in an RGB color space, if the image is 
focused for green wavelengths of light, the same image will tend to be out of focus when 
viewed under blue or red wavelengths of light. Secondly, lateral chromatic aberration is 

25 observed as a difference in magnification for light of different wavelengths due to the 
different focal lengths thereof. For example, in an RGB color space, an image viewed 
under relatively short blue light wavelengths will appear larger than the same image 
viewed under relatively longer red light wavelengths. 
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In microscopy systems having high quality objectives such as, for instance, 
apochromatic objectives, a large portion of the apparent chromatic aberration may be 
corrected. However, some residual lateral chromatic aberration may still remain, 
resulting in differences in magnification across wavelengths of light. This lateral 

5 chromatic aberration may be difficult to visually observe since a human observer tends to 
concentrate on the center of the field of view where the lateral aberration is typically 
absent. However, when imaging the field of view using, for example, a CCD camera, a 
very small lateral chromatic aberration resulting in, for instance, even less that 1% 
difference in magnification between wavelengths, will result in slight color shifts about 

10 the edges of objects in the field of view, but located away for the optical center of the 
objective. Consequently, a pixel located at a given (x,y) position on the image may not 
exactly depict the corresponding portion of the object under investigation depending on 
the wavelengths of light used to illuminate the object and the location of the object within 
the field of view. However, in order to solve chromagen separation equations derived 

15 from the Lambert-Beer law, a basic premise is that the exact same part of the object in the 
field of view must be examined. Therefore, images obtained for separate wavelengths of 
light must be adjusted to provide correlation with respect to the regions of the field of 
view where chromagen separation equations must be solved. 

Accordingly, one advantageous aspect of the present invention involves a method 

20 of correcting lateral chromatic aberration within a microscopy system. First, the 

coordinates of the center of the magnifying objective 250 are determined with respect to 
the center of the electronic device or chip comprising the image-producing component of 
the camera 300. An observed magnification factor is then determined for each 
wavelength and compared to the magnification factor for an arbitrary chosen wavelength. 

25 For example, in an RGB color space, the central wavelength, namely the green channel 
600 would comprise the chosen wavelength to which the magnification factor for each of 
the red 550 and blue 650 channels would be compared. The image for each wavelength 
is then adjusted according to the determined relative magnification factor and the relative 
coordinates of the center of the magnifying objective 250. 

30 In order to facilitate the first two steps of the described method, a specific 

calibration slide is used, wherein the slide is configured with a grid of regularly spaced 
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fine holes through a light blocking media. An image of the grid is taken at each 
wavelength of light used to illuminate the sample. For example, an image may be 
produced for each of the red 550, green 600, and blue 650 channels. The center of each 
hole is then computed in, for instance, x,y coordinates. The image corresponding to the 

5 wavelength of light nearest to the mean of the wavelengths of light under consideration 
(the green channel 600, for example) is then designated as the reference image. 
Subsequently, each of the images for the other wavelength under consideration is then 
compared to the reference image. For each hole in the grid, the difference in the x 
direction (5x) and the difference in the y direction (5y) are then determined for the 

1 0 corresponding hole in the reference image and the image being compared thereto. 

Equations such as, for example, linear equations that minimize the reconstruction error 
for 6x as a function of x and 5y as a function of y, are then determined. From these two 
equations, the center of the objective (x G ,y 0 ) is determined, where x Q comprising the 
solution of the first equation in x when 8x is 0 and y 0 comprises the solution of the 

1 5 second equation in y when 5y is 0. A linear equation that minimizes the reconstruction 
error of 8d, where 8d = (bx^dff 2 , as a function of the distance to the center of the 
objective is then determined, wherein the slope of that equation provides the 
magnification factor of the particular wavelength with respect to the reference 
wavelength. This image for the particular wavelength is then spatially adjusted such that 

20 the origin of the image corresponds to the center of the objective and the magnification of 
the image corresponds to the magnification of the reference image. 

Once the microscope 150 has been configured to provide Keohler illumination for 
image acquisition and chromatic aberrations have been addressed, the additive property 
of the Lambert-Beer law can be applied to chromagen separation. For instance, the 

25 additive property of the Lambert-Beer law can be expanded to a situation in which the 
scene is analyzed in a color environment, generated by, for example, an RGB camera, 
separated into a red, green, and blue channel. In such an instance, the marker dye (or 
"dye 1") exhibits absorption coefficients, € lr , € ig , and €i b , in the red, green and blue 
channels, respectively. Note that, in some instances, the analysis of the image in each of 

30 the red, green, and blue channels is equivalent to analyzing a red representation of the 

image across the red spectra, a green representation of the image across the green spectra, 
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and a blue representation of the image across the blue spectra. Accordingly, the 
counterstain (or "dye 2") exhibits absorption coefficients, e 2r , f2 g , and e 2b , in the red, 
green and blue channels, respectively. Therefore, according to the additive property of 
the Lambert-Beer law, analysis of the sample in the RGB environment leads to three 
5 equations for the optical density thereof: 



where OD r , OD g , and OD b represent the optical densities of the sample measured in the 
red, green and blue channels, respectively. Still further, in the case of increased sample 
preparation complexity such as, for example, the treatment of the sample with three 
different dyes, equations (3), (4), and (5) become: 



1 0 In such a situation, the three dyes may comprise, for instance, one marker dye and two 
counterstains, or two marker dyes and one counterstain, or even three separate marker 
dyes. It will be understood by one skilled in the art, however, that this demonstrated 
property of the Lambert-Beer law may be expanded to included an even greater plurality 
of dye combinations in accordance with the spirit and scope of the present invention. 

1 5 Note also that one particularly advantageous embodiment of the present invention utilizes 
a fast capture color imaging device such as, for example, a 3 CCD RGB camera, for 
multi-spectral imaging of the markers over three distinct (red, green, and blue) channels. 
Accordingly, the exemplary analysis herein is presented in terms of three equations, 
though one skilled in the art will appreciate that the demonstrated concept may be applied 

20 to as many channels as are available with a particular imaging device. 
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OD r = Clr- l\- Q + £ 2 r- h- C 2 

OD g = 6i g • h • Ci + e 2g • h • C 2 
OD b = 6ib • h • Ci + e 2b ' h ' C 2 



(3) 
(4) 
(5) 



OD r = t\ T -h'Ci + e 2r • h • C 2 + e 3r • h • C 3 
ODg = fig • h - Ci + e 2g • h ' C 2+ e 3g • h • C 3 
OD b = cib • h • Ci + € 2b • h ■ C 2+ G 3 b 'h'C 3 



(6) 
(7) 
(8) 



In applying the Lambert-Beer law to a digital microscopy system 100 according 
to embodiments of the present invention, it is difficult and complex, inaccurate, or 
sometimes not possible to measure the thickness 1 of the sample 500. In such instances, 
the concentration C of the molecular specie can be extended and examined as the product 

5 of 1 and C (/ • Q and the results treated accordingly. For example, where the 

concentration of one dye is being compared to the concentration of another dye in a 
particular sample, the sample thickness term will be common to both concentrations and 
thus it becomes less important to determine the sample thickness as an absolute and 
accurate value. Accordingly, it will be understood by one skilled in the art that an 

1 0 accurate determination of the thickness of the sample is typically not required, but may 
generally be treated as a constant in examining the equations as detailed herein. 

The application of the Lambert-Beer law to a digital microscopy system 100 of 
the present invention also recognizes that the Lambert-Beer law can also be expressed as: 

0Z) (x , y) = lOg J 0 (x,y) " lOg Ifcy) (9) 

for a digital image 450 of the sample 500 comprising a plurality of pixels arranged, for 
1 5 example, according to a Cartesian coordinate system, where (x,y) signifies a particular 
pixel in the image 450, OD (x , y) is the optical density of the sample 500 at that pixel, I (x , y) 
is the measured light intensity or transmittance of the sample 500 at that pixel, and I 0 ( X , y ) 
is the light intensity of the light source 200 as measured without any intermediate light- 
absorbing object, such as the sample. Accordingly: 

IOD = EOog / 0(x , y) - log / (x>y) ) (10) 

N 

20 where IOD is the integrated optical density of the digital image 450 of the sample 500, 
and N is the number of pixels in the surface image 450 of the sample. It will further be 
appreciated by one skilled in the art that the logarithmic relationship described in 
equations (9) and (10) may be expressed in various bases within the spirit and scope of 
the present invention. For example, the relationships may be expressed in base 2, base 

25 10, or natural logarithms, wherein the various bases are related by respective 
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proportionality constants (for example, ln(x) or log e (x) = 2.3026 logio(x)). Thus, the 
proportionality constant may be appropriately considered where relative comparisons are 
drawn in light intensities. Further, in quantitative microscopy according to the Lambert- 
Beer law, the proportionality relationship between the optical density OD of the sample 
5 and the dye concentrations is conserved. 

Therefore, for a prepared sample 500 examined by the system 100, the 
appropriate relation is expressed as: 

Inl 0 -Inl = lnl</I=OD = 6'IC (11) 

Where, for example, an 8 bit RGB camera 300 is used in the system 100, the light 
intensity transmitted through the sample in each channel may be expressed as 2 8 (=256) 

10 values between 0 and 255. For example, the initial intensity I 0 of the light source 200, 
which corresponds to 100% transmittance, will preferably be expressed in each of the red 
550, green 600, and blue 650 channels as a value approaching 255, representing the 
brightest possible value in each channel. The camera 300 and/or the light source 200 
may be adjusted accordingly such that, in the absence of the sample, a pure "white" light 

1 5 will have an intensity value of 255 in each of the red 550, green 600, and blue 650 
channels, corresponding to 100% transmittance. Conversely, in the absence of light, 
generally corresponding to transmittance approaching 0%, a "black image" will have an 
intensity value approaching 0 in each of the red 550, green 600, and blue 650 channels. 
At any pixel, the initial intensity I 0 of the light source 200, corresponding to 100% 

20 transmittance, is therefore expressed as the difference between the intensity value 
measured in presence of the light source 200 minus the intensity value measured in 
absence of the light source 200 for each of the red 550, green 600, and blue 650 channels. 
Because the intensity of the light source 200 may vary spatially across the image 450, or 
over the measured field of view, and because the magnifying objective 250 or other 

25 optical components may heterogeneously absorb light, 1 00% transmittance may be 

represented by various differential intensities over the measured field of view. However, 
since the optical density OD of the sample is expressed as the logarithm of the ratio of 
light transmittance in absence of the sample (initial intensity I 0 ) to light transmittance in 

RTA2089257v3 -18- Att'y Docket No. 35735/207206 



presence of the sample (I), the optical density OD is largely spatially insensitive to small 
variations in the differential intensities over the measured field of view. 

Since the light source 200 remains substantially constant over time, or can be 
easily re-evaluated, the measurement of the light intensity for any pixel, in the presence 
of the sample, can be translated into the transmittance I at that pixel and in each of the 
red 550, green 600, and blue 650 channels. Once values for the initial intensity I 0 and 
transmittance I are determined, the optical density OD can be computed. As such, at any 
location in the field of view 450 where a unique dye is present (as the only light- 
absorbing object between the light source 200 and the camera 300), the absorption 
coefficient e of that dye may be determined in each of the red 550, green 600, and blue 
650 channels. More particularly, 1-C for a given pixel will be equal in each of the red 
550, green 600, and blue 650 channels. Thus, if both 1 and C are known, the absorption 
coefficient e can be computed according to equation (11) or in each of the red 550, green 
600, and blue 650 channels as: 

c r = OD r /(l'C) = (ln(I e /I r ))/(l'Q (12) 
€ g = OD g /(lC) = (ln(l os a^)/(l'C) (13) 
e b = OD„/(l-C) = (ln(I ol /I b )) /(l-C) (14) 

However, 1-C is typically not known for a particular pixel in an image of a 
particular sample. Therefore, the absorption coefficients e are computed for each channel 
according to the ratio of the optical density OD in each channel, measured at a given 
pixel, to the maximum optical density OD out of all the channels measured at the same 
pixel. More particularly, it will be appreciated by one skilled in the art that the 
determination of the absorption coefficient e in each of the red 550, green 600, and blue 
650 channels, in the absence of a priori knowledge of 1 and/or C, is a matter of 
manipulating the linear equations in order to achieve a relative solution where 1-C is 
arbitrarily set to a value of 1, wherein: 

€ r = OD r /l =OD r =ln(I 0 /Ir) (15) 
e g = OD g /l =OD g =ln(I og /I g ) (16) 
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e b = 0D b /l = 0D b = ln(I„i/I b ) 



(17) 



Consequently, if the absolute concentration of the particular dye remains unknown, a 
relative absorption coefficient f , in each of the red 550, green 600, and blue 650 channels 
and for any given pixel, may be computed with an error factor equal to l'C. 

Alternatively, because 1 is unique at a given pixel location and can be arbitrarily 
5 set to a value of 1, equations (6-8) may be rewritten as follow where C u C 2 and C 3 are 
related by a factor of 1: 

OD r = €i r ' Ci + 6 2r * C 2 + f 3r * C 3 < 18 ) 

OD g = fig • Ci + e 2g • C 2 + f 3g • C 3 (19) 
OD b = fib * Ci + f 2b ' C 2 + f 3 b • C 3 (20) 

Note that the determination of an absorption coefficient f matrix for different dyes may 
be performed independently of sample evaluation and stored for further application to 
samples treated with at least one of the respective dyes. Further, the various absorption 

10 coefficient f matrices for particular dyes, as well as the original light intensity I 0 data for 
the light source 200 may be stored in, for example, the computer device 350, a server 
located on an intranet or the Internet, or other data storage device as will be appreciated 
by one skilled in the art. As such, when absorption coefficients f have been evaluated for 
the different dyes, and optical densities OD have been determined from image data, the 

1 5 appropriate equations may be solved as a set of linear equations so as to extract the 
respective concentrations of the dyes Ci, C 2 and C 3 . 

By way of further explanation, a representative set of linear algebraic equations 
may be, for example, expressed as: 

anxi + ai 2 x 2 + ai 3 x 3 + . . . + ai N x N = bi 
a 2 iX! + a 22 x 2 + a 23 x 3 + . . . + a 2 NXN = b 2 
a 3 iXi + a 32 x 2 + a 33 x 3 + . . . + a 3 NX>i = b 3 

aMiXi +aM 2 X2 +aM3X 3 + . . . + aMN*N = (21) 
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where, for N unknowns, Xj, j = 1, 2,..., N are related by M equations. The coefficients a ij5 
where i = 1, 2,..., M andj = 1, 2,..., N, are generally known, as are the quantities b s , i = 
1, 2,. . ., M. If M < N, there are effectively fewer equations than unknowns. In such a 

5 case, there can be either no solution or more than one solution matrix x. Further, if N = 
M, then there are as many equations as unknowns and a unique solution matrix x may 
likely be determined, hi addition, if M >N, then there are more equations than unknowns 
and, in general, no particular solution matrix x to the set of equations. Accordingly, the 
set of equations is said to be over-determined and, in such a case, the most appropriate 

1 0 solution is generally considered to be the solution providing the best fit for the equations, 
wherein the best fit solution typically corresponds to the solution having the minimal sum 
of reconstruction errors. 

Equation (21) may also be alternatively expressed as: 



A x = b (22) 



where "'" denotes matrix multiplication, A is the matrix of coefficients, and b is the right 
1 5 side portion expressed as a column vector. Generally, by convention, the first index of an 
element a y denotes the element row; while the second index the element column. 

Further, aj.or a[i] denotes an entire row a[i][j], j=l N. Accordingly, the solution of the 

matrix equation A ' x = b for an unknown vector x, where A is the matrix of coefficients, 
and b is the right side portion, usually requires the determination of A' 1 or the matrix 
20 inverse of the matrix A. Thus: 



x = A 1 b (23) 



Since A" 1 is the matrix inverse of matrix A, then A ■ A" 1 = A" 1 " A = ID, where ID is an 
identity matrix. To facilitate the determination of a solution, parameters may be 
established such that the number of equations is greater than or equal to the number of 
unknowns, or M >N. As previously discussed, when M > N occurs there is, in general, 
25 no particular solution matrix x to equation (21) and the set of equations is over- 

RTA2089257v3 -21- Att'y Docket No. 35735/207206 



determined. In such situations, however, the best "compromise" or best fit solution is 
often the solution that most closely and simultaneously satisfies all of the equations. 
Such closeness may be defined in, for example, a least-squares manner, wherein the sum 
of the squares of the differences between both sides of equation (21) is minimized. As a 

5 result, the over-determined set of linear equations may typically be reduced to a solvable 
linear problem, often referred to as a linear least-squares problem, that may be solved 
with singular value decomposition (SVD) mathematics as will be appreciated by one 
skilled in the art. SVD is directed to the parametric modeling of data and is usually the 
chosen method for solving linear least-squares problems and is described in further detail 

1 0 in, for example, NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 
0-521-43108-5) Copyright (C) 1988-1992 by Cambridge University Press. Programs 
Copyright (C) 1988-1992 by Numerical Recipes Software. 

In some situations, pre-computing solutions for all possible pixel values from the 
described system configuration may effectively facilitate real time processing of the 

15 image analysis. More particularly, if an 8 bit color image acquisition device such as, for 
example, an 8 bit 3 CCD RGB camera is utilized, the measured light intensity I of a 
sample will have 256 possible values ranging between limits of 0 and 255 in each of the 
red 550, green 600, and blue 650 channels. In such an instance, all possible gray values 
(256 3 possible gray values for an 8 bit system) with respect to the original light intensity 

20 I 0 may be pre-computed and stored, for example, as a look-up table (LUT) within the 
computer device 350. Thus, for a sample 500 stained with a particular dye, the 
transmitted light intensity I (or the optical density OD) can be measured at a pixel in each 
of the red 550, green 600, and blue 650 channels and then compared to the previously 
stored gray values and the absorption coefficient e matrix for that particular dye to 

25 thereby determine the dye concentration C, or an estimate thereof as the product 1-C, at 
that pixel. Accordingly, an 8 bit system will provide 256(red channel) x 256(green 
channel) x 256(blue channel) = 256 3 possible gray value solutions, thereby amounting to 
a 16MB LUT for each dye. A system having a gray value resolution exceeding 8 bits per 
channel will lead to larger LUTs such as, for example, a LUT of >1GB for a system 

30 resolution of 10 bits per channel, wherein the computer device 350 may be appropriately 
configured to provide the necessary computing and/or storage capabilities. 
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The operation of the system 100 as described above may be further illustrated by 
example, assuming that the light source is a "white" light having Io = 255 in each of the 
red 550, green 600, and blue 650 channels and that three dyes are used having the 
following transmitted light intensity I characteristics in each of the red 550, green 600, 
and blue 650 channels: 



I 


Red 


Green 


Blue 


Dyel 


168 


127 


94 


Dye 2 


94 


241 


247 


Dye 3 


120 


94 


155 



The corresponding optical density OD matrix (each element being computed as ln(I 0 /I)) 
thus becomes: 



OD 


Red 


Green 


Blue 


Dyel 


0.417 


0.697 


0.998 


Dye 2 


0.998 


0.056 


0.032 


Dye 3 


0.754 


0.998 


0.498 



However, since OD = € • 1 • C, the OD values for each dye can be normalized with 
respect to the channel having the highest OD so as to provide a matrix of relative 
absorption coefficients e for the respective dyes, since the 1 • C values will be constant 
across the channels. Accordingly: 



e 


Red 


Green 


Blue 


Dyel 


0.418 


0.698 


1.000 


Dye 2 


1.000 


0.057 


0.032 


Dye 3 


0.755 


1.000 


0.499 



Subsequently, assuming that a sample 500 has been stained with the same three 
dyes, Dye 1, Dye 2, and Dye 3, and that a light source 200 with similar spectral 
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characteristics is used to illuminate the sample 500, an image 450 of the sample 500 is 
captured by the camera 300. At a particular pixel in the image 450, the computer device 
350 then determines that the transmitted light intensity in each of the red 550, green 600, 
and blue 650 channels is: 





Red 


Green 


Blue 


I 


89 


168 


154 


where: 










Red 


Green 


Blue 


Io 


255 


255 


255 


Thus: 










Red 


Green 


Blue 


OD 


1.053 


0.417 


0.504 



for the particular pixel. Therefore, in order to determine the concentrations of the three 
dyes at that pixel, the OD matrix is multiplied by the inverse of the previously- 
determined relative absorption coefficient e matrix ( (OD)e 1 = 1 • C). Accordingly: 





1*C (mole • cm / L) or C re i a tive 


Dyel 


0.455 


Dye 2 


0.829 


Dye 3 


0.058 



According to the methodology described herein, the determined gray levels or, in 
this example, RGB transmittance values from any combination of the three subject dyes 
may be used to reconstruct an artificial image, since there are no unknowns. 
Accordingly, for that particular pixel and the determined dye concentrations, images for 
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single dyes would correspond to the following Black and White (BW) or RGB pixel 
intensities: 

ln(W) = ln(Io) - ODbw, where OD B w = C (24) 

ln(I r ) = ln(I„) - OD n where OD r = € r C (25) 

ln(I g ) = ln(I 0 ) - OD g , where OD g = e g C (26) 

\n(I b ) = ln(I 0 ) - OD b , where OD b = e b -C (27) 

Accordingly: 
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Intensity (I) 


Dye 


Evaluated 


BW 


Red 


Green 


Blue 


Dyel 


0.455 


161 


210 


185 


161 


Dye2 


0.829 


111 


111 


243 


248 


Dye3 


0.058 


240 


244 


240 


247 



Further advantageous aspects of the present invention are realized as a result of 
5 the dye separation techniques using color video imaging as previously described herein. 
For example, an artificial image of the field of view may be generated in an RGB color 
space or in gray levels as a substantially real time or live image, or as a still image, using 
combinations of the dyes comprising a marker and/or a counterstain used to prepare the 
sample. More particularly, an artificial image of the field of view may be produced 
10 which shows the sample as affected by all of the dyes, the sample as affected by one or 
more marker dyes, or the sample as affected by the counterstain. Consequently, since the 
dyes used to prepare the sample are characterized by the system, the capabilities of the 
system may be extended such that, for instance, the sample or field of view may be 
automatically scanned to detect a specific region of interest as identified by the 
15 characteristics of a particular dye or to affect or facilitate a task to be performed on that 
specific region of interest. 

According to one embodiment of the present invention, the system may be 
configured so as to be capable of detecting one or more particular dyes which have been 
previously characterized by the system. In some instances, such a dye may comprise, for 
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example, the ink from a particular pen or similar ink marker that has been characterized 
by the system as having unique color features, these unique color features being retained 
by the system as a corresponding set of extinction coefficients. It follows that the system 
may be configured to recognize and respond to portions of the field of view in which this 

5 dye is identified and that, in some instances, the one or more particular markers may 
comprise a tangible portion of such a system as described herein. For instance, such a 
pen may be used, for example, where an operator such a pathologist or a cytotechnologist 
identifies special areas of interest on a sample-containing glass or plastic slide. A special 
area of interest may comprise, for example, a potential diagnostic area or a reference 

1 0 area. The operator, using the pen, may then surround the area with a line of ink from that 
pen. After processing a number of slides, the operator may feed the slides into, for 
instance, an automatic scanning system for quantitative evaluation. Having been 
configured to detect the ink from the pen, the system may then inclusively identify the 
area of interest, corresponding to the area within the ink line, circled by the operator with 

15 the pen. The system may thereafter appropriately process that area of the slide where, for 
example, one color of pen ink may indicate that a particular diagnostic evaluation must 
be performed, while another color of ink may indicate that the area contains a calibration 
or reference material and would call for the system to run a corresponding calibration 
procedure. Note that, in addition to slides, the described technique may be readily 

20 adapted to examine other mounting forms for microscopic material such as, for example, 
microtiter plates or microarrays. Thus, it will be appreciated by one skilled in the art that 
the capabilities of such embodiments of the system, configured to recognize particular 
dyes or inks, may extend to many different automatic scanning processes where 
interactive marking of areas of interest with specific pens, the pens having different color 

25 inks previously evaluated by the system, may be used to automatically designate and 
actuate a subsequent evaluation or other processing of that area of interest by an 
appropriate component of the system or other specified device. 

Additionally, the artificial images of the field of view may also facilitate the 
presentation of the data in a configuration allowing identification and selection of 

30 meaningful objects or areas of interest as, for example, still images in a report prepared 
for diagnostic or other reporting purposes. 
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Other advantageous aspects of the present invention may also be realized from the 
described system and method herein. For instance, the differences in characteristics 
between the marker dyes and the counterstain, as realized in various dye-specific images 
of the sample, may be used to evaluate the focus adequacy of the field of view. More 
particularly, for example, the sample may be treated with two separate dyes, one dye 
comprising a nuclei stain and the other dye comprising a membrane stain. In such an 
instance, an image directed to the membrane stain may be evaluated for focus adequacy 
by examining the focus of the same image directed to the nuclei stain, wherein the nuclei 
stain image exhibits a more definite structure upon which evaluate focus. 

Still further, the artificial image of the field of view may also be used to facilitate 
the identification and extraction of selected features of the treated sample. For example, 
marked point processes, contextual analysis, and/or geo-statistics may be used to identify 
and extract features from the image based on, for instance, a spatial distribution analysis 
of a particular dye. Such a feature extraction capability would also allow, for example, 
fields of view or objects of interest to be sorted, flagged, or otherwise identified or 
grouped based on, for instance, the overall content of a given marker dye or a selected 
ratio of particular marker. Where, for example, a threshold criteria can be established, 
such a capability would be the detection of rare, worsening, or other serious events. 
Proceeding further, classifiers based specifically on the image processing resulting from 
the counterstain and/or marker dye specific images may then be established and used to 
evaluate the presence of certain cell types or to perform a diagnosis based upon the field 
of view. For example, HER2 may be evaluated in this manner by comparison to a 
continuous diagnosis scale established according to the system and methods described 
herein. Such classifiers may usually also encompass other informative features such as, 
for example, detail based upon the morphology or the texture of the cells. 

Still further, another advantageous aspect of the present invention is realized 
where the system is capable of processing the image data at a faster rate than the images 
are acquired. The enhanced speed at which the image data is processed may allow, for 
example, features indicated by a particular marker dye to be processed and classified. 
Accordingly, various conditions may be identified based upon predetermined criteria. As 
such, visual and/or sonic alarms maybe established and/or mapped in conjunction with 
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the processing of the image data. Thus, in some instances, the operator's attention may 
be directed to a specific field of view or object of interest when a characteristic of a 
marker attains a predetermined level in, for example, intensity or presence in a particular 
field. 

5 FIG. 2 is a schematic representation of a practical realization of an extended 

system configuration according to one embodiment of the present invention. In such an 
implementation, the system 100 or workstation is centered about a microscope 150. The 
microscope 150 may include one or more robotic components including, for example, a 
motorized stage, an automatic focus mechanism, a motorized objective changer, and an 

10 automatic light intensity adjustment. The system 100 may also include various input 
devices such as, for instances, cameras 300a and 300b having fast automatic focusing 
and configured for acquiring low-resolution and high-resolution images, a flat bed linear 
scanners 310 used for acquiring low-resolution images, a grossing station 320, and a 
voice-recording device 330, which are all linked to a computer device 350 through 

15 various data transmission links 400. The workstation 100 can be part of a Local Area 
Network (LAN) 700, but may also be configured to support different communication 
protocols such that available communication channels such as, for example, a standard 
telephone line, an ISDN connection, or a Tl line, can readily connect the workstation 100 
to other components or devices over large distances via a Wide Area Network (WAN) 

20 750 as will be appreciated by one skilled in the art. 

If the pathology workstation 100 is configured to operate in an integrated 
environment, the WAN 700 or LAN 750 connection may permit access to, for instance, 
existing reference databases 800 and Hospital Information Systems (HIS) 850. With 
such a configuration, new samples and/or cases may readily be compared with the 

25 pictures and accompanying information of previously-accumulated reference cases. 
Further, images acquired from the samples and/or slides being examined at the 
workstation 100 can be complemented with the patient and case history as necessary. 

In the extended configuration embodiment as shown in FIG. 2, the pathology 
workstation 100 is particularly configured for a comprehensive sample evaluation. For 

30 example, with information and digital pictures of the initial gross biological sample, 
images of the slides prepared from the sample can be prepared and processed as 
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described herein. The patient and case information, the images, and the resulting 
quantitative information about the cell components of the sample and the sample 
architecture (in the case of, for instance, tissue samples) can collected, integrated if 
necessary, and stored in a single database. If, for example, an initial or second expert 
5 opinion is needed or if the slide is used for training or proficiency testing, the 

communication capabilities of the extended configuration along with the automation 
features of the microscope 150 may allow the workstation 100 to be used as a tele- 
pathology system. For example, high-resolution images directed to features or objects of 
interest characterizing a questionable situation on a particular slide may be electronically 

10 forwarded to the expert and/or to the audited candidate. In some instances, an overview 
picture of the slide may be provided, wherein the automated microscope 150 is used to 
scan the slide automatically on, for example, a field by field basis. The corresponding 
digital images may then be stored in the memory of the computer device 350. Where a 
field by field basis is used, the edges of adjacent fields may be precisely matched using 

15 correlation algorithms, so as to provide a single large overview image of the entire slide. 
Such an overview image may assist the reference pathologist in making an assessment of 
the information. In some instances, the reference pathologist may remotely control the 
workstation 100 from a remote site to acquire necessary and/or supplemental images 
which may be required so as to provide a correct and thorough assessment of the slide. 

20 Subsequently, the information accumulated by the workstation 100 for a studied 

case such as, for instance, real or mathematically generated images, measurement results 
and graphical representations thereof, patient data, preparation data, and screening maps, 
may be selectively integrated into a report which can either be printed or accessed 
electronically. Such a report would provide a comprehensive picture of the case under 

25 evaluation and would also facilitate quality assurance and standardization issues. 

It will be understood that the methodology and procedures detailed herein in 
conjunction with the system 100 specify a method of quantifying an amount of a 
molecular specie from an image of a sample captured by an RGB camera in a video- 
microscopy system. One skilled in the art will also appreciate that such a method may be 

30 automated so as to provide a computer software program product, executable on a 
computer device, having executable portions capable of quantifying the amount of a 
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molecular specie from a digital image of a sample captured by a color image acquisition 
device, such as an RGB camera, in a video-microscopy system. Accordingly, 
embodiments of the system 100 describe the implementation of the method and/or the 
corresponding computer software program product which may be accomplished in 
5 appropriately configured hardware, software, or a combination of hardware and software 
in accordance with the spirit and scope of the present invention. 

Thus, embodiments of the present invention comprise a colorimetric analysis 
technique for prepared samples that provides effective detection and quantification of 
species of interest that overcomes limiting factors of prior art techniques such as, for 

10 example, spectral overlapping, mixing of colors due to spatial overlap of membrane and 
nuclear markers, limited spectral resolution of the acquisition device, calibration 
particularities, the subjectivity of the detection and quantification process, and 
inconsistencies between human operators of the analysis equipment. Embodiments of the 
present invention further provide an image processing technique which does not rely 

15 upon the subjective detection of contrast within the prepared sample or a complex and 
voluminous analysis of the sample at specific wavelengths of light using a combination 
of light sources and filters. Therefore, embodiments of the present invention provide a 
simpler and more effective colorimetric analysis technique that overcomes detection and 
quantification limitations in prior art analysis techniques, reduces subjectivity and 

20 inconsistency in the sample analysis, and is capable of providing the necessary analysis 
information about the sample, once an image of the sample is captured, without relying 
upon further examination of the sample to complete the analysis. 

More particularly and as demonstrated, the analysis (detection and quantification 
of a molecular specie of interest) of the prepared sample is accomplished through the 

25 measurement of light intensities that are manifested in a digital image of the sample 
captured by a color image acquisition device. Since the analysis is relatively image- 
dependent, rather than sample-dependent, redundant images may be captured for 
analysis, while many samples may be processed so as to capture the necessary images 
within a relatively short period of time. Once the image data has been captured and 

30 stored, the actual analysis may occur at a later time or as needed without requiring the 
physical presence of the actual sample. Such an analysis may be further applied to 
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examining the entire sample or even the entire slide. Thus, embodiments of the present 
invention provide an expeditious quantitative video-microscopy system that permits the 
use of such a system as a routine or "production" tool capable of accomplishing a 
relatively high analysis throughput. As such, significant advantages are realized by 
5 embodiments of the present invention as compared to prior art quantitative microscopy 
systems which were typically limited in sample throughput and analysis, thus generally 
making such systems more useful as research tools. 

Many modifications and other embodiments of the invention will come to mind to 
one skilled in the art to which this invention pertains having the benefit of the teachings 
10 presented in the foregoing descriptions and the associated drawings. Therefore, it is to be 
understood that the invention is not to be limited to the specific embodiments disclosed 
and that modifications and other embodiments are intended to be included within the 
scope of the appended claims. Although specific terms are employed herein, they are 
used in a generic and descriptive sense only and not for purposes of limitation. 
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